Convergence Analysis of the Information Potential Criterion in Adaline Training

In our recent studies we have proposed the use of minimum error entropy criterion as an alternative to minimum square error (MSE) in supervised adaptive system training. We have formulated a nonparametric estimator for Renyi’s entropy with the help of Parzen windowing. This formulation revealed interesting insights about the process of information theoretical learning. We have applied this new criterion to the training of linear and nonlinear adaptive topologies under the problems of blind source separation, channel equalization, and time-series prediction with superb results. In this paper, we analyze the structure of the entropy criterion performance surface around the optimal solution and we derive the upper bound for the step size in Adaline training with the steepest descent algorithm. We also investigate the effects on adaptation of the kernel size in the Parzen windowing, and order of Renyi’s entropy.